m 


SunQenics 

mm?  Coiporation 


Effectiveness  of  Systems  Engineering  (SE)  Tailored 
for  the  Science  &  Technology, (S&T)  Environment: 
Improvement  of  USAF  Airdrop  Accuracy 

14th  Annual  NDIA  Systems  Engineering  Conference 

24-27  October2011 


Carol  Ventresca 

Carol@SynGenics.com 


Case  Number:  88ABW-201 1 -5353,  Distribution: 
Approved  for  public  release;  distribution  is  unlimited 


Form  Approved 
OMB  No.  0704-0188 


Report  Documentation  Page 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

OCT  2011 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2011  to  00-00-2011 

4.  TITLE  AND  SUBTITLE 

Effectiveness  Of  Systems  Engineering  (SE)  Tailored  For  The  Science  & 
Technology  (S&T)  Environment:  Improvement  Of  USAF  Airdrop 
Accuracy 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

SynGenics  Corporation, San  Antonio, TX, 78248- 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

Presented  at  the  National  Guard  Bureau  Joint  C4I  Conference,  Nashville,  Tennessee,  6-9  Dec 
2010, Government  or  Federal  Purpose  Rights  License 

14.  ABSTRACT 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF  PAGES 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

35 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Coauthors 


□  Dr.  Keith  Bowman,  AFRL,  Precision  Airdrop  (PAD) 
Program  Manager 

□  Ms.  Carol  Ventresca,  SynGenics  Corporation 

□  Mr.  Robert  McCarty,  SynGenics  Corporation 

□  Ms.  Stephanie  Globus,  Syngenics  Corporation 


Case  Number:  88ABW-201 1 -5353,  Distribution: 
Approved  for  public  release;  distribution  is  unlimited 


2 


Background 


□  USAF  Aerial  Delivery  Operations  Increasing  Dramatically 

□  Drove  Need  to  Improve  Accuracy  for 

>  Critical  Resupply 

>  Humanitarian  Aid 

□  AMC  Requested  AFRL  Investigate  Technology  Solutions 

>  Aid  Development  of  Systems  to  Achieve  AMC  Need 

>  Many  Complexities  Drove  Need  for  Systems  Engineering 
r  S&T  SE  Process  Drove  FY12  AFRL  Technology  Investment 

>  Multiple  Technology  Projects  Planned  in  2011 
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considered..."  Gen  Raymond  Johns, 
Commander  AMC,  2011 
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Entrance  Criteria  for 


Airdrop 


□  Documented/Prioritized  MAJCOM  Capability  Gap 

□  Commissioned  Via  AF  S&T  Governance  Structure 

□  Linked  to  Service  Core  Function  Master  Plan 

□  Initial  S&T  Development  Strategy  Initiated 

□  Between  a  Leading  Development  Planning  Concept 
and  a  Prototype 

□  Assigned  to  Lead  Center  for  Transition 

□  MAJCOM  Transition  Manager  Identified 

□  Defined  S&T  Baseline/Exit  Criteria 

□  S&T  Activity  Ideally  Completed  During  Current  FYDP 
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Integrated  Product  Team 


□  The  Precision  Airdrop  (PAD)  IPT 

>  Air  Force  Research  Laboratory  (AFRL) 

>  Air  Mobility  Command  (AMC) 

>  US  Army  NATICK 

>  Electronic  Systems  Center  (ESC) 

>  Aeronautical  Systems  Center  (ASC) 

>  US  Air  Force  Academy  (USAFA) 


SynGenics 

mm  mm  Corporation 


Case  Number:  88ABW-201 1-5353,  Distribution: 
Approved  for  public  release;  distribution  is  unlimited 


8 


Presentation  Outline 


□  Entrance  Criteria  for  PAD 

□  Integrated  Product  Team  (IPT) 

□  S&T  SE  Process  Steps 

□  Initial  Project  S&T  Development  Strategy 

□  User  Understanding  of  Desirements 

□  Products  from  S&T  SE  Process 

□  Categories  of  Candidate  Technology  Options 

□  Techniques  to  Score  Solution  Options 

□  Methods  to  Combine  Options  into  Alternatives 

□  Methods  to  Score  Alternatives 

□  Findings  from  Application  of  S&T  SE  Process 

□  Critical  Roles  Played  by  S&T  SE  in  Pre  Milestone  A 


Case  Number:  88ABW-201 1 -5353,  Distribution: 
Approved  for  public  release;  distribution  is  unlimited 


S&T  SE  Process 


Step  4: 
Evaluation 


Step  3: 
Alternatives , 


*  Develop  Desirements  and  Metrics 

-  Solicit  Input  from  All  Stakeholders 

-  Define  Measurands,  Desirability 
Functions,  and  Relative  Importance 

-  Repeat  as  Knowledge  Advances 


9  Generate  Technology  Alternatives  and  Conceptual  Designs 
9  Perform  Value  Analysis  to  Evaluate  Alternatives 

—  Evaluate  Alternatives  against  Desirements 

—  Compute  Desirability  and  Risk  for  Each  Concept 

—  Explore  Trade  Space 

—  Generate  or  Refine  Alternative  Approaches 

—  Select  Most  Promising  Approach 

9  Deliver  Results:  Recommend  Alternatives 
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Initial  Schedule 


Schedule*  for  Systems  Engineering  (SE)  Support  to  an  AFRL  Future  Capability  Concept  Precision  Airdrop  (PAD)  Program 

Version  2.5  as  of  1 2  Oct  1 0 


SE 

Activities 


SEP 

OCT 

NOV 

DEC 

JAN 

FEB 

20  |  27 

4  |  11  |  18  |  25 

1  |  8  |  15  |  22  |  29 

6  |  13  |  20  |  27 

3  |  10  |  17  |  24  |  31 
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£  US  Army  Natick  &  AMC  possibly  feeding  ICD  or  OCR  for  upgrades  to  existing  systems 
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XUS  Army  Natick  &  AMC 

•  ▲ 

****  AFRL  SE  support 


t US  Army  Natick  &  AMC 

•  A 

AFRL  SE  support 


|US  Army  Natick  &  AMC 

:  AFRL  SE  support  including  DOE  for  sensitivity  studies  and  MS&A  for  Natick 


Legend: 

Notes: 


AO  Complete 


Open 


This  schedule  will  he  maintained  as  a  living  document.  SE  Activities  need  to  he  iterative  across  swim  lanes/  not  serial. 
Possible  System-of-Systems  approach,  building  requirements  at  that  level,  and  then  hacking  off  to  focus  on  PAD 
AFRL  Team  will  look  beyond  a  single  S&T  solution:  will  address  long-term  ramifications  regarding  cost  and  schedule 
AFRL  Team  will  start  small,  engage  with  SMEs.  visibility  for  Directorate  Chief  Engineers,  possibly  include  RB.  RX.  RY  . 


Meetings  supported  by  Syn Genies  and  MacAulay-Brown 
(Meetings  2-5  are  2-3  days.  J 

1.  AFRL  Core  Team  (inci.  3-4  SMEs)  at  Scott  AFB  with  AMC 

and  US  Army  to  discuss  an  AFRL  FCC  for  PAD  and  to  agree  on 
means  of  providing  a  "single  voice"  to  AFRL  for  both  services. 

2.  AFRL  Core  Team  at  WPAFB  with  AMC  and  Army  Natick  for  2 
days  of  Syn  Genies  -facilitated  PAD  Requirements  Development 

3.  AFRL  Core  Team  at  Scott  AFB  with  AMC  and  Army  Natick 
for  SynGenics -facilitated  development  of  PAD  Exit  Criteria 

4.  AFRL  Core  Team  at  Scott  AFB  with  AMC  and  Army  Natick 

for  SynGenics -facilitated  Development  of  Concepts/Alternatives 

5.  AFRL  Core  Team  at  Scott  AFB  with  AMC  and  Army  Natick 
for  SynGenics -facilitated  Analysis  and  Tradeoffs 


f\  Products 

1.  Joint  AMC  /  US  Army  PAD  Requirements 

2.  Joint  AMC  /  US  Army  PAD  Exit  Criteria 

3.  Full  Set  of  C o nc e pt s/Alt em at ives  (tech  and  operational) 

4.  Analysis  and  Tradeoffs  vs  AMC/Army  PAD  Desirements 

5.  AFRL  Flagship  Capability  Concept  (FCC)  Baseline  for  PAD 
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-,f  Precision  Airdrop  (PAD)  Focus  Areas 


□  Precision  was  the  Original  Focus  of  the  PAD  Project 

□  AMC’s  Desire  was  for  AFRL  to  Address  Urgent  Needs  in 

>  Critical  Resupply 

>  Humanitarian  Airdrop 

□  These  Urgent  Needs  Shaped  the  Definition  of  “Precision” 

>  Precision  was  Viewed  only  as  Impact  Point  Accuracy 

□  The  PAD  Project  now  Addresses  Precision  as 

>  Single  Pass 


>  Dispersion  Predictability  and  Tailorabilil 

>  Situational  Awareness  of  Bundles 

>  Impact  Point  Accuracy 

>  Predictability  in  the  Event  of  Malfunctioi 
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Eight  S&T  SE  Process  Deliverables 


□  Documented  Criteria:  Includes  “Exit  Criteria” 

□  Alternatives:  Potential  Solution  Concepts  Captured,  Defined,  and 
Assessed  Against  all  Criteria 

□  Analysis:  a  Mathematically  Based  Evaluation  of  Alternatives,  Including 
Quantified  Predictions  of 

>  Response  Values  Related  to  Criteria 

>  Desirability,  Uncertainty,  and  Risk 

□  Sensitivity  Analysis:  Reveals  Highly  Leveraged  Parameters  Through 
Exploitation  of  their  Acceptable  Ranges 

□  Relationships: 

>  Between  Factors  and  Responses 

>  Among  Desirements 

□  Understanding:  the  Process  Demands  that  all  Desirements  be  Satisfied 
and  the  Solution  “Trade  Space”  be  Understood 

□  Worksheets  and  Scorecards:  Framework  for  Presentation  of  Results  & 
for  Revisiting  Them  when  New  Information  Emerges 

□  Consensus 
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For  FBS  Elements 


Current 

Focus 


□  Payload/Exit  Improvements 


□  Communication/Display  Improvements 

□  Weather  Data  Acquisition  Improvements 
^  Human  Factors  Mitigation 

□  UAV  Integration 

□  Additional  Studies 
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Scoring  of  Options 

Initial  Assessment 


Des 

# 

Desirement  Name 

Units 

Current 

1-Skid 

1-SkidAdv 

1-Dun 

1-DunAdv 

1-Release 

Active  Shaping 

Force  Ex 

Air  Bags 

Expecte(iA/or/Bs1ixpecte(iA/or/Bs1:xpecte<iA/or/Bs1ixpecte(iA/or/Bs1:xpecte(iA/or/Bs1!xpecte(iA/or/Bs1:xpecte(iA/or/Bs1ixpecte(iA/or/Bs1:xpecte(iA/or/Bs1 

Category:  A.  Performance 

P01 

Impact  Point  Accuracy 

meters 
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325 
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300 

675 

400 
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400 

800 
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175 
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175 
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meters 
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87.5 

287.5 

87.5 
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Execution 
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Scale: 
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Scoring  of  36  options 
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completed  13  Dec 
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ining  Options  into  Alternatives 


□  Generated  Alternatives  as  All  Combinations 
of  Options  in  Accordance  with  Rules 
Established: 

>  Every  Alternative  Had  at  Least  One  Option 
from  Each  Type 

>  No  More  than  Two  Weather  Options  Were 
Allowed 
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Scoring  Alternatives 


□  Scored  Each  Alternative  against  Desirements  Using 
Worst,  Best,  or  Multiplicative  Rules  Applied  to  Scores 
of  Options  Present 

□  Analyzed  Customer  Desirability  of  19,530  Alternatives 
thus  Generated 

□  Inspected  Top  5,000  Alternatives  (Type  of  Pareto 
Analysis) 

□  Generated  Scorecards  for  Customer  Desirability  and 
Risk  for  Top  12  Alternatives 

□  Identified  Alternatives  that  Offer  Greatest  Chances  for 
PAD  Improvements 
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Findings 


□  No  Single  Option  Solved  the  Entire  Problem 

>  Hence  the  Need  to  Evaluate  Alternative  System  Solutions 

□  No  0-3-yr  Option  Addressed 

>  PI  4,  Minimum  Mass  Delivery  Capability 

>  HF5,  Rigging  Workload 

>  HF6,  Rigging  Training  Required 

□  Doing  Less  Is  Superior  for  Human  Factors  Desirability 

>  Can  Only  Hurt  Security 

□  Risk  Analysis  Is  Suspect  Because  of  Scoring  Concerns 


SynGenics 

mm  mm  Corporation 


Case  Number:  88ABW-201 1-5353,  Distribution: 
Approved  for  public  release;  distribution  is  unlimited 


27 


Options  in  Best  Alternatives 
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Frequency  of  Occurrence  of  0-3-yr 
Options  in  Top  5,000  Alternatives 

*0^"  ^Performance  01"  ^Overall 


Option 

Type 

Perf 

Overall 

HALO 

i 

91.6% 

82.1% 

Snipe 

2 

72.2% 

33.2% 

HSCDS 

1 

67.1% 

71 .9% 

ASACMS 

2 

66.7% 

70.7% 

CommNow 

2 

66.6% 

56.6% 

AELVIS 

2 

65.8% 

51 .4% 

1-Dun 

1 

59.0% 

59.5% 

1-Release 

1 

57.5% 

57.5% 

1-Skid 

1 

55.1% 

51.1% 

Wedge 

2 

50.0% 

50.1% 

MAF-ADTgt 

2 

49.9% 

46.2% 

UAV-Wx 

3 

43.1% 

39.1% 

LIDAR-ABW 

3 

43.0% 

38.7% 

EnsembleFC 

3 

39.8% 

48.4% 

Radar-Wx 

3 

39.2% 

34.0% 

SunGenics 
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Cost  Sensitivity? 


□  Contribution  to  “Goodness”  When  More  Options  Are 
Allowed  within  an  Alternative 


□  Contribution  When  Only  3  or  4  Are  Permitted 


l 


o 


■  Performance 

■  Overall 
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Way  Forward  (as  of  29  sep  11) 


□  Completed  Assessment  of  Alternatives  Scorecard 

■  Identified  Alternatives  Offering  Greatest  Chances  for 
Improvement  to  PAD 

■  Briefed  Results  to  the  AFRL/AMC  Team,  Initiating  the 
IPT  Planning  Phase  (Exit/Comm/Weather/Human  Focus) 

□  Alternative  IPTs  Refine  Alternatives  and  Define  Tech 
Path  Forward  (28  Jan) 

□  Interim  Review  with  AFRL/CC  (28  Feb) 

□  Integrated  Baseline  Review  Completed  (27  Sep) 

□  IPT  for  Each  Focus  Area  is  in  Place  to  Commence 
Execution  of  FY12-16  Plan 
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Summary 


□  The  Systems  Engineering  (SE)  Approach  is  an 
“ Eye-Opening ”  Experience... Making  Us  Think 
Outside  What  We  Already  Knew 

□  A  Cross-AF/Service/TD  Team  was  Formed 

>•  Met  >  Weekly  to  Capture/Refine  Desirements  and 
Generate/Evaluate  Solutions 

□  The  AFRL-Employed  S&T  SE  Process  is  Generating 
New  Thinking  to  Solve  a  Critical  AF  Need 
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Observations  by  Leaders 


□  AMC,  by  Col  Peet,  AMC/A8X,  in  a  Message  to  Dr. 
Erbschloe,  AMC/ST: 

"...we  think  all  this  work  is  great,  and  will  inform  future 
efforts  also.  So,  definitely  keep  this  scoring  methodology. 
We  do  find  great  value  in  it." 

□  AFRL,  by  PAD  Project  Lead: 

>  The  Process  Broadened  Scope  of  Analysis  to  Include 

■  Traditionally  Army-Owned  Pieces  of  the  Problem 

■  Very  Near-Term  Technology  Options 

>  A  Detailed  FBS  of  the  Airdrop  Problem  Revealed  Issues 
that  Would  Have  Been  Overlooked  had  a  SE-Based 
Approach  not  been  Employed. 
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Contact  Information 


Carol  Ventresca 
Phone  740  369-9579 
Cell  Phone  614  668-8300 
SynGenics  Corporation 

carol@svnqenics.com 
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